Skip to content

Conversation

@mariegrho
Copy link
Collaborator

short version of the quickstart, first draft

@mariegrho mariegrho requested a review from flo-schu May 8, 2025 12:00
@mariegrho mariegrho linked an issue May 8, 2025 that may be closed by this pull request
@flo-schu flo-schu changed the base branch from main to 100-documentation-for-hierarchical-modelling May 8, 2025 16:27
@flo-schu flo-schu changed the base branch from 100-documentation-for-hierarchical-modelling to dev May 8, 2025 16:28
Copy link
Owner

@flo-schu flo-schu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mariegrho

the super-qiuckstart looks already very good :) I've many commented on some of the wording. It might be good if you check for typos at the end.

Thanks for the great work!

2. __Model:__
Our model will be defined as a python function.
We will then assign it to our Simulation object by `.model`

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: "We will then assign it to our Simulation object by accessing the .model attribute"


3. __Observations:__
Our observation data needs to be structured as a [xarray.Dataset](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html).
We assign it to our Simulation object by `.observations.`
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: "We assign it to the .observations attribute of our simulation object"


4. __Solver:__
Solvers are needed to solve the model.
In our simple case, we will use the solver "solve_analytic_1d" from the "pymob.solver.analytic module.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe mention, that the solve_analytic_1d solver basically does nothing, because the model already contains the solution. It is only needed to satisfy pymob, which requires a solver

We assign the parameters to our Simulation object by `sim.model_parameters`. This is a dictionary that holds the model input data. The keys it takes by default are `parameters`, `y0` and `x_in`.

7. __Evaluator:__
The Evaluator is an instance to evaluate a model.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. is before 6.

Pymob supports our modeling process by providing several tools for *structuring our data*, for the *parameter estimation* and *visualization of the results*.

Before starting the modeling process, we let's have a look at the main steps and modules of pymob:

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about another heading here "pymob components" or similar?

data_obs = xr.DataArray(y_obs, coords={"t": t}).to_dataset(name="y")
data_obs
```

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice and clear! :)


## Estimating parameters

We are almost set infer the parameters of the model. We add another parameter to also estimate the error of the parameters, We use a lognormal distribution for it. We also specify an error model for the distribution. This will be
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are almost set to infer...

...parameters and use a lognormal distribution for it.

Our model is now prepared with a parameter set.
In order to intialize the *Evaluator* class, we need to execute `sim.dispatch_constructor()`.
This step is very important and needs to be done everytime when we made changes in our model.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... after making changes to our model

Comment on lines 1103 to 1104
sim.config.model_parameters.b.min = -5
sim.config.model_parameters.b.max = 5
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The min/max parameters are not needed when running inference with numpyro, so I suggest to remove them, otherwise it might be confusing

Comment on lines 1174 to 1177
### Estimating parameters and uncertainty with MCMC

Of course this example is very simple, we can in fact optimize the parameters perfectly by hand. But just for the fun of it, let's use *Markov Chain Monte Carlo* (MCMC) to estimate the parameters, their uncertainty and the uncertainty in the data.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would rather fit before the previous block.

@flo-schu
Copy link
Owner

flo-schu commented Jun 17, 2025

Hi @amelieleo, thanks for uploading the introduction. I like it a lot :) Of course I still have a few remarks:

  • base module
    • SolverBase
    • curve_jumps
    • jump_interpolation
    • mappar
    • radius_interpolation
    • rect_interpolation
    • smoothed_interpolation

Please remove the base module from 4. solver. This only contains utility functions and the Base parent class for the other solvers

Note: If you want to rename your data-dimension you have to change every .data to the new name!

This is not quite clear to me. What does .data refer to here? Ah I get it know. Maybe it helps to make it clear that .data refers to the data variable observations["data"] or sim.config.data_structure.data

Case studies are a principled approach to the modelling process. In essence, they are a simple template that contains building blocks for model and names and stores them in an intuitive and reproducible way. Here you'll find some additional information oon case studies.

typo "oon" (highlighted)

Running the model 🏃

The model is prepared with a parameter set and ready to be executed. With sim.dispatch_constructor(), everything is prepared for the run of the model. It initiaizes an evaluator, makes preliminary calculations and checks.

I would like to add a dispatch_constructor info-box:
"What does the dispatch constructor do? Behind the scenes, the dispatch constructor assembles a lightweight Evaluator object from the Simulation object, that takes the least necessary amount of information, runs it through some dimension checks, and also connects it to the specified solver and initializes it. The purpose of the dispatch constructor is manyfold:
By executing the entire overhead of a model evaluation and packing it into a new Evaluator instance sim.dispatch_constructor() to make single model evaluations as fast as possible and allow parallel evaluations, because each evaluator created by sim.dispatch() is it's a fully independent model instance with a separate set of parameters that can be solved.
Evaluators store the raw output from a simulation and can generate an xarray object from it that corresponds to the data-structure of the observations with the evaluator.results property. This automatically aligns simulations results with observations, for simple computation of loss functions."

Manual estimation

...
from matplotlib import pyplot as plt
def plot(results: xr.Dataset):
obs = sim.observations

We could also add a loss-function which is just the SSE = ((results.data - obs.data) ** 2).sum(dim="time"). and add it's value as a text to the plot

ax.text(0.05, 0.95, f"SSE={np.round(SSE.values, 2)}", transform=ax.transAxes, ha="left", va="top") <- there may be a typo in this command.

This could be nice for people, to see how results and obs are automatically aligned.

The meanof b is the value of the estimated parameter. It shloud be the same or close to estimation you did manually. The sigma_y is the mean error of this estimation.

typo "shloud"

Use of API references

I also have a general remark regarding cross references to the API. @mariegrho @amelieleo @merkuns: When you write the guides, please use as often as reasonably possible direct API references wherever possible. E.g.:

{class}`pymob.simulation.SimulationBase`

instead of SimulationBase only

(use {class} for classes, and {meth} for methods; see https://www.sphinx-doc.org/en/master/usage/domains/python.html#cross-referencing-python-objects). The difference is that with the former, links to the respective API documentation will be automatically inserted. The path is the full path to the respective class @amelieleo, @mariegrho can you look over the documents and replace class and method mentions with references?

Copy link
Owner

@flo-schu flo-schu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is complete @mariegrho @amelieleo 🎉
For @merkuns tutorial we will open a separate PR

@flo-schu
Copy link
Owner

flo-schu commented Jul 28, 2025

@merkuns thanks for providing the first version of the ODE system tutorial. I like it a lot!

One thing I noticed when I tried to build the documentation is that the script cannot be executed as a whole (which is required for automated documentation deployment). This is because you build in errors, which I like a lot. The way to work with this is to use a try-except block catching the error you want to produce and then printing it as a message without raising an error. You could also raise an error if the normal function call does not produce an error 🤪

To try to build and preview the documentation and check whether this runs without errors, you can execute the commands in the docs/README.md sections 'Using jupyter notebooks for the documentation' and 'Compile documentation'. These should run without errors

Some other comments

  • add the pymob.sim.parse_input() as an admonition (e.g. hint, note, warning, ...) block. You can have a look at the previous tutorials for this.
  • In 1.5 add also this note as an admonition
  • Before chapter 2, you can add a hint admonition, urging the users to take a look at the automated report which is compiled to the output directory
  • Use unix compatible paths in 2.1 "/" instead of "\"
  • I like the way you point at a case study tutorial. Maybe you could give a short peek at subclassing at the end, to show that this can be indeed quite easy
class LotkaVolterraSim(SimulationBase):
    model = lotkavolterra
    solver = JaxSolver
    def initialize(self, input=None):
        super().initialize(input)
        self.model_parameters["parameters"] = self.config.model_parameters.value_dict
        self.dispatch_constructor()
    

sim3 = LotkaVolterraSim("case_studies/ODEtutorial/scenarios/lotkavolterra/settings.cfg")
sim3.initialize()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve the Quickstart Guide and add advanced tutorials

5 participants